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ABSTRACT 



This paper presents the results of the first-year evaluation 
of Bridging the Digital Divide in South Florida, a 3 -year Preparing 
Tomorrow's Teachers To Use Technology grant. Over the 3 years of this 
project, elementary school teachers from two Florida counties, college 
professors, and preservice teachers from two Florida universities will work 
collaboratively to integrate technology into classrooms and design curriculum 
with lesson plans to assist in this implementation. The report focuses on the 
first stage of the integration process in the 2000-2001 academic year. The 
evaluation team used multiple sources of data, including online instruments, 
surveys, chat-room transcripts, and pen-and-pencil evaluations. For the first 
year, eight collaborative subject area teams were formed to include 
professors, teachers, and preservice teachers in each. Each of the teams was 
assigned a discussion board to which participants contributed postings. 
Findings from this first year indicate that the project focuses on providing 
technology training to preservice teachers, and that the resources necessary 
have been available in the first year. Project timelines were met throughout 
the year, and participants were successful in conducting project activities 
in the virtual team format in which they did not meet face-to-face. The 
program was implemented as was intended, but some recommendations were made 
for project improvement. These center on consistency in recording program 
activities and results. (SLD) 
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Integrating Technology in the Classroom 
Using Virtual Teams 

The purpose of this paper is to present the results of the first year evaluation of 
bridging the Digital Divide in South Florida,” a three-year Preparing Tomorrows 
Teacher’s to Use Technology (PT3) grant. These grants provide financial support for 
innovative programs designed to prepare teachers to use technology to improve student 
learning. Over the course of the three years of this project, elementary teachers from 
Miami-Dade and Broward Counties, college professors and pre-service-teachers from 
two South Florida universities will work collaboratively to (1) integrate technology into 
elementary school classrooms, and (2) design curriculum (lesson plans) to assist in this 
implementation. This report focuses on the results of stage one of the integration process 
covering the 2000-2001 academic year. 

Grounded in the goal of infusing technology instruction into all aspects of 
(1) elementary school education and (2) pre-service teacher education training, the 
program articulated objectives in eight areas: the general elementary curriculum, English 
as a Second Language (ESOL), Exceptional Student Education (ESE), General Teaching 
Methods courses by creating an on-line learning community to include in-service K-12 
educators. Math, Reading/Language Arts, Science, and Social Studies. For year one, 
eight collaborative subject area teams were formed to include professors, K-12 teachers, 
and pre-service teachers. These teams worked together in person and online to integrate 
technology into their classrooms. 




3 



2 



Evaluation Design 

A mixed method evaluation design was used to answer the questions related to the 
project goal and to assess outcomes related to program effectiveness. This design utilized 
qualitative and quantitative methodologies to triangulate results. This triangulation 
strengthens the evaluative findings (Breen, Jenkins, Lindsay, & Smith, 1998). 

The quantitative methodology used in this evaluation provided data for assessing 
actual use of technology and skill assessment and attainment (e.g.. Profiler, LoTi, mid- 
year survey). The qualitative methodology provided additional data on perceptions 
related to the implementation of the project. In addition, analysis of small numbers of 
texts and documents (e.g., on-line postings and chat room discussions) allowed for a 
better understanding of how these texts depict reality in a given circumstance (Silverman, 
2000 ). 

Theoretical Framework 

In addition to an evaluation design, a theoretical framework was used to guide the 
evaluation of the project. It provided a structure for thinking about data and a basis for 
generalizing the qualitative findings (Seale, 1999). 

An Interactive Evaluation Model (Owen & Rogers, 1999) specifically, a 
Responsive Evaluation Design (Stake, 1980), was utilized for this project. This 
evaluation design focuses on program activities and provides feedback and information 
for program change and improvement. Responsive Evaluation is intended to help assist 
stakeholders in achieving a better understanding of the how the project is functioning 
(Owen & Rogers, 1999). 
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Validity cmd Reliability 

To increase validity the evaluation team used multiple sources of data. Analysis 
of chat room transcripts was used to help understand how the team leaders perceived 
what was happening with their teams. Surveys were used to obtain information on how 
the participants viewed the program effectiveness. Workshop evaluations gave insight 
into what the participant found useful and effective, and skill and integration surveys 
helped provide baseline data on the abilities of the participants. Triangulation of data was 
obtained through analysis of these multiple sources of data (Stake, 1995). 

Insuring that data collection methods were consistent throughout the project and 
between the evaluators increased reliability. In addition, member checks (Stake, 1995) 
were conducted by distributing chat room transcripts and mid-year evaluation reports to 
participants for verification. Participants were also encouraged to review their on-line 
technology survey results (Profiler, LoTi). Finally, reliability of the qualitative findings 
was addressed using standard procedures in the field such as defining the reporting of the 
methods and the theoretical framework that guides the study, and using multiple 
evaluators (Seale, 1999). 

Evaluative Questions 

Six evaluative questions were developed to guide this evaluation. The evaluative 
questions are grounded in the research framework used for this study, the Responsive 
Evaluation Model that focuses on the process of the project implementation. 

1 . What is the project trying to achieve? 

2. Is the project designed to meet the objectives? 

3. Are the resources available to implement the project as planned? 
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4. Is the delivery consistent with the project plan? 

5. What is the current status of the project? 

6. How could the project be changed to make it more effective? 

Data Sources 

As reported above multiple sources of data collection were used for this 
evaluation. These included on-line instruments and surveys, chat room transcripts, and 
pen and pencil evaluations. 

Profiler. Profiler is a web-based survey developed by the High Plains Regional 
Technology in Education Consortium. The instrument provides K-12 educators with the 
opportunity to assess their technology literacy. Two questionnaires were set up on the 
Profiler website for project participants to take when they began the project and again at 
year’s end. The two questionnaires on the Profiler website can be found at 
http :/www. profiler.org under the headings: (1) STU-PT3 Survey #1 - Technical Skills 
and (2) STU-PT3 Pedagogical Uses of Technology. 

Level of Technology Implementation (LoTi). Another on-line questionnaire, LoTi, 
was administered to project participants at the beginning of the project. This instrument 
was selected to evaluate participants’ level of technology integration. LoTi casts items in 
three domains: (1) LoTi - Level of Technology Implementation, (2) PCU - Personal 
Computer Use, and (3) CEP - Current Instructional Practices. 

Mid-year Participant Survey \ In order to determine participants’ involvement and 
active participation in grant activities, the evaluators designed an 8-question survey that 
included both closed and open-ended responses. 




5 



Activity Logs. Project participants were required to keep a PT3 Professional 
Development Log to record project-related activities. Recorded in the log was the date, 
description of the activity, and time spent in the activity. The activities were coded as: (1) 
Partner Workshops, (2) Optional Workshops, (3) On-line Training, (4) Conferences, (5) 
Mentors, (6) Tech Support, and (7) Individual Training. The project director developed 
the log with input from the team leaders. 

Workshop Evaluations. As project participants completed each workshop, they 
were asked to complete a workshop evaluation form developed by the project evaluators. 

Team Leader Chats. Team leaders and the project director conducted three on-line 
chat sessions during the program. Transcripts of two of these chat sessions were recorded 
and distributed to participants. They present a record of the team leaders perceptions of 
the project. 

Discussion Board Postings. 

Each of the eight teams was assigned a discussion board where participants were asked to 
contribute substantive postings throughout the year. Descriptive data on the usage and 
topics of these di scussion boards were analyzed to determine the frequency and relevance 
of postings. 

Results 

Evaluative Question 1: What is the project trying to achieve? 

The grant’s focus is to provide technology training to pre-service teachers in the 
partnership universities by infusing technology into the pre-service teacher’s subject area 
curriculum. Subsequently, the eight project objectives and three project outcomes 
previously presented were articulated. 




7 



6 



Evaluative Question 2: Is the project designed to meet its objectives? 

A copy of the project was submitted for an expert review of the program plan. 

The following is a quote from that review. “I have no critical observations about the way 
the project is structured. It looks very solid with clearly attainable goals given the three- 
year plan.” 

Evaluative Question 3: Are the resources available to implement the project as planned? 

Various resources were necessary for effective delivery of the project including 
personnel resources and the areas of administrative and support responsibilities, structure 
of the implementation teams, technical capabilities of team members, facilities; and 
finances. Each of these resources has been successfully addressed in year one of the 
project. 

Evaluative Question 4: Is the delivery consistent with the project plan? 

The project timelines were met throughout the year. Face to face activities were 
planned and attended. And participants were successful in conducting project activities in 
formats where they do not have face-to-face contact, through email and postings on the 
WebCT bulletin boards. Participants have relied on resource personnel throughout the 
first year of the project. The administrative assistant has facilitated a communication flow 
and has maintained documentation of team member activities. The computer technician 
has maintained the computer lab and aided participants in software acquisition and use. 
The web-site coordinator has developed and maintained both the project web site, to 
coordinate and disseminate project activities, and the WebCT site to facilitate team 
communications. 
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Evaluative Question 5: What is the current status of the project? 

The program was fully implemented as it was originally intended. The PT3 Grant 
Performance Report submitted by the project director indicated all of the objectives for 
year one had been met and technology had been successfully integrated into pre- service 
teacher instruction in the eight team areas. Additionally, all of the project GPRA 
indicators were met for this year. 

Participants were required to maintain activity logs to document their 
participation in the project. The activity in which team members reported spending the 
most time was Partner Workshops with an average of 5.9 hours per activity, followed by 
Conferences with an average of 5.7 hours per activity and Other Workshops with an 
average of 4.3 hours per session. 

To encourage discussion and interaction among the team members, on-line 
bulletin boards were created on WebCT. Together with their Activity Logs, team 
members were asked to submit transcripts of their postings to the team bulletin board. 

The average number of “substantive” postings for the year is 26.8. 

During the second semester, monthly team leader meetings were held. These face- 
to-face team leaders meetings gave the participants an opportunity to meet with the 
project director, support staff, and evaluation team to share ideas, express concerns, and 
plan for the following year. 

Twice during the second semester, the team leaders met on-line with the project 
director. The transcripts were qualitatively analyzed. Three key themes emerged from an 
analysis of the January transcripts (1) technical/task issues, (2) pedagogical issues, and 
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(3) social issues (team building). The categories that surfaced in the February chat were 
(1) technical/task issues, (2) program improvement issues, and (3) accountability issues. 
Evaluative Question 6: How could delivery be changed to make it more effective ? 

j 

Because the evaluators had an on-going and active role in the development of the 
program over the past year, they were aware of the value perspectives of those involved 
in the project. It is with this in mind that recommendations were offered to address the 
final evaluative question. 

Four recommendations focused on consistency in recording activities. Two 
recommendations addressed communication within teams and with project personnel and 
team leaders. Five recommendations addressed assessment, particularly the development 
and use of instruments to measure technology proficiency, interest, and use. Two 
recommendations were made that the grant be amended, as several performance 
indicators were not perceived as being available or as appropriate to be used as indicators 
of program effectiveness. There were two personnel recommendations, including the 
addition of a liaison at one of the universities with responsibilities for facilitating 
communication and disseminating information. The other personnel recommendation 
encouraged the participants to continue using the skills and the commitment to the project 
that resulted in the successful implementation of year one. 

Educational Importance of the Study 

Integrating technology into the school curriculum has been likened to Dorothy’s 
trip down the yellow brick road. You are sure to face stormy conditions, experience a few 
wicked spells here and there, meet friends to share the journey with, and at the end find 
that all of the answers lie within (Irwin-Robinson & Robinson, 2000). This paper reports 
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the (1) successful completion of program components designed to integrate technology 
into elementary school education and the pre-service teacher curriculum and (2) identifies 
various elements of the program that can be modified to make the project even better. 
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